PIXEL COMPENSATING CIRCUIT, METHOD FOR COMPENSATING PIXELS, 
AND IMAGE TAKING APPARATUS EMPLOYING SUCH PIXEL 
COMPENSATING CIRCUIT 

5 BACKGROUND OF THE INVENTION 

1. Field of the Invention 

This invention relates to a pixel compensating circuit, a method for 
compensating pixels, and an image taking apparatus employing such pixel 
compensating circuit, wherein defect pixels included in a video signal obtained from 
10 an image sensor are compensated. 

2. Description of the Related Art 

In a digital still camera and a video camera, a CCD (Charge Coupled Device) 
image sensor or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor 
is widely used in a solid state image taking apparatus. 

15 In such image sensor, defect pixels which do not function well are included 

sometimes among pixels of the image sensor. For example, when such image sensor 
product is carried by an airplane, there is a problem, wherein some of pixels of the 
image sensor are sometimes damaged by the passage of a cosmic ray having high 
energy. Those image sensors having such defect pixels are wasted at assemble stage. 

20 However, this causes high cost in total, so that various methods for 

compensating defect pixels have been considered so far. For example, there are a 
sampling hold method where a value of a pixel immediately before (immediate left) a 
defect pixel is employed as the value for the defect pixel, an averaging method where 
an average value of two pixels located immediately before and immediately after the 

25 defect pixel is employed, and the like. Further, there is another method called an 
edge detecting method where edge information is detected by taking differences 
between color information of the defect pixel and color information of adjacent pixels 
located immediately after the defect pixel, and the compensation is carried out from 
defect pixel which is thought to be small in its effect. 

30 However, in the above mentioned sampling hold method and the averaging 

method, there is a problem wherein possibility of carrying out a significant erroneous 
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compensation are high when abrupt edges overlap on the defect pixel, and adequate 
compensation result can not be expected. Particularly, even when two or more than 
two pixels in a row have defects, the compensation by the above mentioned methods is 
carried out uniformly, so that there is a risk of standing out the defect pixel by contrast. 
5 From this point of view, although the edge detecting method is one of excellent 
compensating methods, but the edge detecting method has a problem in which when 
the compensation is erroneously carried out, the defect pixel may have a color different 
from periphery because the edge detecting method carries out the compensation by 
focusing on only one color information. 

10 

SUMMARY OF THE INVENTION 
This invention is carried out in consideration of the above mentioned point, 
and an aspect of the present invention is to propose a pixel compensating circuit for 
compensating defect pixels, and an image taking apparatus employing such pixel 

15 compensating circuit. Further, another aspect of the present invention is to propose a 
pixel compensating method for compensating defect pixels. 

In order to solve the above mentioned problems, the present invention 
proposes a pixel compensating circuit for compensating defect pixels included in a. 
video signal, wherein the pixel compensating circuit comprises: a color information 

20 holding unit for holding plural kinds of color information of a defect pixel and pixels 
adjacent the defect pixel; a difference calculating unit for calculating differences 
between zero defect color information among the color information of the defect pixel 
and the color information of the pixels adjacent the defect pixel corresponding to the 
zero defect color information; a reference pixel determining unit for determining a 

25 reference pixel having color information that is the most similar to the zero defect 
color information; and a compensating unit for calculating an average value of the 
difference for the reference pixel and the defect pixel, and for compensating the zero 
defect color information of the defect pixel using the average value. 

According to the above mentioned configuration, a reference pixel which is 

30 the most similar to a defect pixel is determined, and then defect color information is 
compensated using an average value of the difference between the defect pixel and the 
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reference pixel. 

Further, the present invention propose an image taking apparatus employing 
the above mentioned pixel compensating circuit for compensating defect pixels. 

Still further, the present invention presents a pixel compensating method for 
5 compensating a defect pixel of a video signal, comprising the steps of: holding plural 
kinds of color information of defect pixel and pixels adjacent the defect pixel; 
calculating differences between zero defect color information among the color 
information of the defect pixel and the color information of the pixels adjacent the 
defect pixel corresponding to the zero defect color information; determining a 
10 reference pixel having color information that is the most similar to the zero defect 
color information; calculating an average value of the difference for the reference pixel 
and the defect pixel; and compensating the zero defect color information of the defect 
pixel using the average value. 

According to the above mentioned method, a reference pixel which is the most 
15 similar to a defect pixel is determined, and then defect color information is 
compensated using an average value of the difference between the defect pixel and the 
reference pixel. 

BRIEF DESCRIPTION OF THE DRAWINGS 
20 In the accompanying drawings: 

Fig. 1 is a functional block diagram showing a fundamental principle of a 
pixel compensating circuit according to the present invention; 

Fig. 2 is a chart showing one example of held color information; 
Fig. 3 is a chart showing a condition wherein a sum of absolute values of 
25 differences is calculated; 

Fig. 4 is a block diagram sowing one example of an image taking apparatus; 
Fig. 5 is a flowchart showing one embodiment of a method for compensating 
pixels according to the present invention; 

Fig. 6 is a chart showing one example of held color information per pixel; 
30 Fig. 7 is a chart showing one example where the number of pixels for 

calculation is reduced; 
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Fig. 8 is a chart showing one example where three defect pixels are existed in 
a row; and 

Figs. 9 A to 9C are charts showing adjacent pixels for calculation in case of 
compensating defect where three defect pixels are existed in a row, wherein Fig. 9A 
5 shows a compensation of a left pixel, Fig. 9B shows a compensation of a center pixel, 
and Fig. 9C shows a compensation of a right pixel, respectively. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
One embodiment of the present invention will be described with reference to 
10 accompanying drawings hereinafter. Fig. 1 is a functional block diagram showing a 
fundamental principle of a pixel compensating circuit according to the present 
invention. 

A pixel compensating circuit 10 of the present invention comprises of a color 
information holding unit 11, a difference calculating unit 12, a reference pixel 

15 determining unit 13, and a compensating unit 14. ^ 

The color information holding unit 1 1 holds plural kinds of color information 
for a defect pixel and adjacent pixels having no defect with reference to a position and 
defect information such as a color for the defect pixel stored in a defect information 
holding unit 20. More specifically, as shown in Fig. 1, outputs from image sensors 

20 21a, 21b, and 21c provided for each of plural colors such as R (Red), G (Green), and B 
(Blue) are held as plural kinds of color information per color. 

Fig. 2 is a chart showing one example of the held color information. Here, 
for the purpose of simplifying the description, only a defect pixel 0 having defect in 
green color GO, and adjacent pixels 1 through 5 are depicted, wherein those adjacent 

25 pixels are respectively positioned at left side, right side upper side, upper-left side, and 
upper-right side. The color information holding unit 11 holds values of R, G, and B 
which are color information of the defect pixel 0, and the adjacent pixels 1, 2, 3, 4, and 
5. For example, regarding the defect pixel 0, the color information holding unit 11 
holds a value of 59 for the color R0, a value of 250 for the color GO, and a value of 48 

30 for the color B0 with respect to each color. The color information holding unit 11 
also holds values of colors Rl through R5, values of colors Gl through G5, and values 
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of colors Bl through B5 as the color information with respect to all of the adjacent 
pixels 1 through 5. 

The difference calculating unit 12 calculates the differences between the color 
information without defect among color information of the defect pixel and the same 
5 color information of each of the adjacent pixels. For example, in the case of Fig. 2, 
the differences between values of the color RO and the color BO, that are not defect 
colors, among the color information of the defect pixel 0 and values of colors Rl 
through R5, and Bl through B5 of the adjacent pixels 1 through 5 are calculated with 
respect to respective colors. 
10 The reference pixel determining unit 13 determines one of the adjacent pixel 

as a reference pixel for compensation, where sum of the absolute values of the 
differences calculated at the difference calculating unit 12 becomes a minimum with 
respect to each color. 

The compensating unit 14 calculates an average value of the differences for 
15 the reference pixel, and compensates the defect color information of the defect pixel 0 
based on the calculated average value. 

Hereinafter, an operation of the pixel compensating circuit 10 is explained 
with reference to Fig. 1 and Fig. 2. 

When color information is inputted through the external image sensors 21a, 
20 21b, and 21c, as shown in Fig. 2, the values of R, G, and B are held in the color 
information holding unit 11 with respect to the defect pixel 0 and each of the adjacent 
pixels 1 through 5 having no defect. Next, at the difference calculating unit 12, the 
differences between the values of colors RO and BO without defect among the color 
information of the defect pixel 0 and the values of the colors Rl through R5, and the 
25 colors Bl through B5 of the adjacent pixels 1 through 5. Further at the reference 
pixel determining unit 13, the adjacent pixel where the sum of the absolute values of 
the differences becomes minimum is selected. 

Fig. 3 is a chart showing a condition wherein the sum of absolute values of the 
differences is calculated. 
30 In this case, a reference code dr designates the respective differences between 

the values of the colors Rl through R5 for the adjacent pixels 1 through 5 and the 
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value of color RO for the defect pixel 0, while a reference code db designates the 
respective differences between the values of the colors Bl through B5 for the adjacent 
pixels 1 through 5 and the value of color BO for the defect pixel 0. In addition, the 
sum of the absolute values of the differences is designated as | dr | + | db | . As 
5 being understood from Fig. 3, the sum of the absolute values of the differences for the 
adjacent pixel 1 becomes 34, and this becomes minimum among the sums for other 
adjacent pixels 2 through 5. From this fact, it is judged that the adjacent pixel 1 is the 
one having the color information that is the most similar to the color information of the 
defect pixel 0, and this adjacent pixel 1 is determined as a reference pixel. 

10 In this case, though neglected in the above description, a weighing unit which 

applies a coefficient to the differences depending on the distance between the defect 
pixel 0 and the target adjacent pixel may be provided between the difference 
calculating unit 12 and the reference pixel determining circuit 13. 
Specific explanation will be described later. 

15 Next, one embodiment of a pixel compensating circuit of the present invention 

is described in detail. 

Hereinafter, one embodiment of the pixel compensating circuit of the present 
invention is described in a case where the pixel compensating circuit is applied to a 
color imaging apparatus (hereinafter referred to as an image taking apparatus) having 

20 image sensors of CCD, CMOS, and the like. 

Fig. 4 is a block diagram showing one example of an image taking apparatus. 
An image taking apparatus 100 comprises of an optical lens system (herein 
after referred to as a lens) 101 for inputting an image, a color separating optical system 
(hereinafter referred to as a prism) 102 for separating color information into respective 

25 color of R for red, G for green, and B for blue, image sensors 103a, 103b, and 103c 
provided corresponding to each color of R, G, and B, a CDS (Correlated Double 
Sampling) circuit 104 for eliminating noises, an AGC (Automatic Gain Control) circuit 
105 for adjusting a value of an output signal, a LPF (Low Pass Filter) circuit 106 for 
eliminating a superfluous signal, a WB/AMP (White Balance/Amplifier) circuit 107 

30 for adjusting a white balance, an AD (Analog-Digital) converting circuit 108 for 
carrying out an AD conversion, a pixel compensating circuit 109, an EPROM 
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(Erasable Programmable Read-Only Memory) 110 for holding the defect information, 
and a DSP (Digital Signal Processor) 111 for processing compensated color 
information. 

Incident lights from the lens 101 are separated into R, G, and B at the prism 
5 102, and are irradiated on respective image sensors 103a, 103b, and 103c. These 
lights are converted into electrical signals by these image sensors 103a, 103b, and 103c, 
and these electrical signals are converted to digital values at the AD converting circuit 
108 after passing through the CDS circuit 104, the AGC circuit 105, the LPF circuit 
106, and the WB/AMP circuit 107. Output signals from the AD converting circuit 

10 108 are inputted to the pixel compensating circuit 109 corresponding to the present 
invention. The pixel compensating circuit 109 properly carries out the compensation 
of defect pixels based on the defect information such as a defect position, a defect 
length, a color having defect, and the like recorded in the EPROM 110. Hereinafter, a 
pixel compensating method of the present invention is described while explaining the 

15 operation. 

Fig. 5 is a flowchart showing one embodiment of a method for compensating 
pixels according to the present invention. After starting at step START, values of R, 
G, and B for the defect pixel and the adjacent pixels having no defect are held at step 
SI. In this embodiment, the values of color R, color G, and color B for the defect 
20 pixel and those of the adjacent pixels are held based on the defect information recorded 
in the EPROM 110. 

Fig. 6 is a chart showing one example of the held color information per pixel. 
As shown in this Fig. 6, values of color R0, GO, and B0 for the defect pixel 0 and the 
adjacent pixels 1 through 8 are held. As to the defect pixel 0, for example, the values 
25 of color R0, GO, and B0 are held, while as to the pixel 1, values of color Rl, Gl, and 
Bl are held. Similarly, values of respective R, G, and B are held up to the pixel 8. 

A reference pixel in which the sum of the absolute values of the differences 
becomes minimum is determined at step S3. Here, the sums of the absolute values of 
the differences per color calculated at step S2 are obtained by a following equation. 
30 A x = | Bx - B0 | + | Gx - GO | , where x varies from 1 to 8. 

One of adjacent pixels, wherein the sum of the absolute values of the 
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difference among thus calculated sums Al through A8 by the above equation 
corresponding to the pixels 1 through 8 is the minimum, is judged as a pixel having the 
most similar color information (R,G, and, B) to that of the defect pixel 0, and is defined 
as a reference pixel to be used for compensation. 
5 The defect color information of the defect pixel 0 is compensated based on the 

average values of the differences for the reference pixel at step S4. 

When one of adjacent pixels, wherein the sum of the absolute values of the 
difference among thus calculated sums Al through A8 corresponding to the pixels 1 
10 through 8 is the minimum, is defined as the reference pixel x at step S3, the 
compensation is carried out by making the color R0 of the defect pixel 0 to be the 
value that cause to coincide the average values of the differences for the color G and B 
with the difference for the color R. 

Specifically, it is calculated by a following equation. 
15 (| Gx-G0| +| Bx-B0| )/2 = Rx - R0 

Accordingly, R0 = Rx - (| Gx - GO | + | Bx - BO | )/2, wherein the Rx, Gx, 
and Bx are color information for the reference pixel x. 

As described above, it is able to carry out the compensation of the defect color 
R0 for the defect pixel 0. 
20 In this case, it is possible to automatically carry out the proper compensation 

based on the suitable peripheral or adjacent pixels by continuously operating the 
present compensating method. 

In addition, if there is also a defect in any of colors Gl through G8 of the 
pixels 1 through 8, it is possible to carry out the compensation by applying a 
25 conventional averaging method, a method of coinciding only the difference of the color 
B with the difference of the color R (wherein the color G is ignored), and a method of 
calculating without such defect pixel 0. 

Further, although it is rare case, if there also exists a defect in the color GO of 
the defect pixel 0, it is able to carry out the compensation by applying the above 
30 mentioned averaging method, and the pixel compensating method while excluding the 
color GO. 



According to the present invention, it is necessary to hold data such as color 
information of the defect pixel and adjacent pixels per defect. 

Fig. 7 is a chart showing one example where the number of pixels for 
calculation is reduced. 

5 As described above, when it is necessary to reduce the data capacity for 

holding the color information, the calculation may be carried out using the pixels of 
left, right upper, and lower sides relative to the defect pixel 0 as shown in Fig. 7, for 
example, without using all adjacent pixels 1 through 8 relative to the defect pixel 0. 

Then, the case where the length of the defect is 2 pixels or more is described. 
10 In the case where the length of the defect is 2 pixels or more, the pixel 

compensating method of the present invention may be applied per each defect pixel. 

Fig. 8 is a chart showing one example where three defect pixels are existed in 

a row. 

In the case where defects exist in 3 pixels in a row such as the defect pixels 0a, 
15 Ob, and 0c, the color information of the adjacent pixels 1 through 12 is held. In the 
case of compensation, the compensation may be carried out while deciding the 
adjacent pixel to be applied to the calculation per one pixel as described below. 

Figs. 9A to 9C are charts showing adjacent pixels for calculation in case^ of 
compensating defect where three defect pixels are existed in a row, wherein Fig. 9A 
20 shows a compensation of a left pixel, Fig. 9B shows a compensation of a center pixel, 
and Fig. 9C shows a compensation of a right pixel, respectively. 

In case of compensating the defect pixel 0a on the left side, it is able to 
compensate the defect color information of the defect pixel 0a with the defect 
compensating method as shown in Fig. 5 by calculating the differences between the 
25 color information of pixels 1, 2, 3, 6, 8, 9, and 10 and the color information other than 
the defect color information of the defect pixel 0a. 

Further in case of compensating the defect pixel Ob located at center, it is able 
to compensate the defect color information of the defect pixel Ob using the color 
information of pixels 2, 3, 4, 6, 7, 9, 10, and lias shown in Fig. 9B, and further, in case 
30 of compensating the defect pixel 0c on the left side, it is able to compensate the defect 
color information of the defect pixel 0c using the color information of pixels 3, 4, 5, 7, 



10, 11, and 12 as shown in Fig. 9C. 

Further, in the case of Fig. 9B, the pixels 6 and 7 are located farther from the 
defect pixel Ob than the other pixels 2, 3, 4, 9, 10, and 11. Accordingly, it is thought 
that the pixels 2, 3, 4, 9, 10, and 11 have higher correlation with the defect pixel Ob 
5 than the pixels 6 and 7. Therefore, in case of compensation, the calculation is carried 
out by doubling the difference between the color information of the defect pixel Ob and 
the color information of the pixels 6 and 7. 

In addition, when driving the CCD using an interlaced scanning, data of the 
upper side and the lower side of the defect are in a state where they are aligned away 

10 from the defect pixel by one pixel to be precise. This is similar to the interlacing of 
the television system. In the odd field, pixels on 1st, 3rd, 5th , ... line are employed, 
and pixels on 2nd, 4th, 6th , . . . line are employed in the even field. In this situation, 
it is so considered that the pixels on left and right sides of the defect pixel Ob may have 
higher correlation with the defect pixel Ob than the pixels on upper and lower sides of 

15 the defect pixel, so that the calculation may be carried out using the difference between 
the color information of the defect pixel Ob and the color information of the pixels 6 
and 7 as it is. 

As described above, it is able to carry out a precise compensation to the 
defects of continuing 2 or more pixels. In the above embodiment, the colors R, G, 
20 and B are employed for description, but they are not limited to them, and the colors of 
cyan, magenta, and yellow may be employed and they may be more than three colors. 
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